SemEval 2014 Task 8: Broad-Coverage Semantic Dependency Parsing

We define broad-coverage semantic dependency parsing (SDP) as the task of recovering sentence-internal predicate–argument relationships for all content words, i.e. the semantic structure constituting the relational core of sentence meaning. 1 Background and Motivation Syntactic dependency parsing has seen great advances in the past decade, in part owing to relatively broad consensus on target representations, and in part reflecting the successful execution of a series of shared tasks at the annual Conference for Natural Language Learning (CoNLL; Buchholz & Marsi, 2006; Nivre et al., 2007; inter alios). From this very active research area accurate and efficient syntactic parsers have developed for a wide range of natural languages. However, the predominant data structure in dependency parsing to date are trees, in the formal sense that every node in the dependency graph is reachable from a distinguished root node by exactly one directed path. Unfortunately, tree-oriented parsers are ill-suited for producing meaning representations, i.e. moving from the analysis of grammatical structure to sentence semantics. Even if syntactic parsing arguably can be limited to tree structures, this is not the case in semantic analysis, where a node will often be the argument of multiple predicates (i.e. have more than one incoming arc), and it will often This work is licenced under a Creative Commons Attribution 4.0 International License. Page numbers and the proceedings footer are added by the organizers: http:// creativecommons.org/licenses/by/4.0/. be desirable to leave nodes corresponding to semantically vacuous word classes unattached (with no incoming arcs). Thus, Task 8 at SemEval 2014, Broad-Coverage Semantic Dependency Parsing (SDP 2014),1 seeks to stimulate the dependency parsing community to move towards more general graph processing, to thus enable a more direct analysis of Who did What to Whom? For English, there exist several independent annotations of sentence meaning over the venerable Wall Street Journal (WSJ) text of the Penn Treebank (PTB; Marcus et al., 1993). These resources constitute parallel semantic annotations over the same common text, but to date they have not been related to each other and, actually, have hardly been used for training and testing of datadriven parsers. In this task, we have used three different such target representations for bi-lexical semantic dependencies, as demonstrated in Figure 1 below for the WSJ sentence: (1) A similar technique is almost impossible to apply to other crops, such as cotton, soybeans, and rice. Semantically, technique arguably is dependent on the determiner (the quantificational locus), the modifier similar, and the predicate apply. Conversely, the predicative copula, infinitival to, and the particle marking the deep object of apply can be argued to not have a semantic contribution of their own. Besides calling for node re-entrancies and partial connectivity, semantic dependency graphs may also exhibit higher degrees of non-projectivity than typical syntactic dependency trees. In addition to its relation to syntactic dependency parsing, the task also has some overlap with SeSee http://alt.qcri.org/semeval2014/ task8/ for further technical details, information on how to obtain the data, and official results. A similar technique is almost impossible to apply to other crops , such as cotton , soybeans and rice . A1 A2 (a) Partial semantic dependencies in PropBank and NomBank. A similar technique is almost impossible to apply to other crops, such as cotton, soybeans and rice. top ARG2 ARG3 ARG1 ARG2 mwe _and_c ARG1 ARG1 BV ARG1 implicit_conj ARG1 (b) DELPH-IN Minimal Recursion Semantics–derived bi-lexical dependencies (DM). A similar technique is almost impossible to apply to other crops , such as cotton , soybeans and rice top